State Aware Imitation Learning
نویسندگان
چکیده
Imitation learning is the study of learning how to act given a set of demonstrations provided by a human expert. It is intuitively apparent that learning to take optimal actions is a simpler undertaking in situations that are similar to the ones shown by the teacher. However, imitation learning approaches do not tend to use this insight directly. In this paper, we introduce State Aware Imitation Learning (SAIL), an imitation learning algorithm that allows an agent to learn how to remain in states where it can confidently take the correct action and how to recover if it is lead astray. Key to this algorithm is a gradient learned using a temporal difference update rule which leads the agent to prefer states similar to the demonstrated states. We show that estimating a linear approximation of this gradient yields similar theoretical guarantees to online temporal difference learning approaches and empirically show that SAIL can effectively be used for imitation learning in continuous domains with non-linear function approximators used for both the policy representation and the gradient estimate.
منابع مشابه
Learning to Search via Self-Imitation
We study the problem of learning a good search policy. To do so, we propose the self-imitation learning setting, which builds upon imitation learning in two ways. First, self-imitation uses feedback provided by retrospective analysis of demonstrated search traces. Second, the policy can learn from its own decisions and mistakes without requiring repeated feedback from an external expert. Combin...
متن کاملALS Seminar - Imitation Learning
Machine Learning is an important tool for developing complex systems. In particular, Reinforcement Learning has been successfull in learning robot behavior. However, it can be difficult to define appropriate reward functions. Especially in robotics, one has to deal with high dimensionality in the state/action space. Imitation of experts is a fundamental part of learning in nature. Imitation, in...
متن کاملEmbodied imitation-enhanced reinforcement learning in multi-agent systems
Imitation is an example of social learning in which an individual observes and copies another’s actions. This paper presents a new method for using imitation as a way of enhancing the learning speed of individual agents that employ a well-known reinforcement learning algorithm, namely Q-learning. Compared to other research that uses imitation with reinforcement learning, our method uses imitati...
متن کاملActive Imitation Learning via State Queries
We consider the problem of active imitation learning. In passive imitation learning, the goal is to learn a target policy by observing full trajectories of it. Unfortunately, generating such trajectories requires substantial effort and can be impractical in some cases. Active imitation learning reduces this effort by querying the teacher about individual states. Given such a query, the teacher ...
متن کاملAgile Off-Road Autonomous Driving Using End-to-End Deep Imitation Learning
We present an end-to-end imitation learning system for agile, off-road autonomous driving using only low-cost on-board sensors. By imitating a model predictive controller equipped with advanced sensors, we train a deep neural network control policy to map raw, high-dimensional observations to continuous steering and throttle commands. Compared with recent approaches to similar tasks, our method...
متن کامل